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THE INTRODUCTION 

In the late summer of 1968, the Evaluation and Research 
Center for Project Head Start at the University of South Carolina 
began its third year of evaluating year-round Head Start programs 
in the Southeast. The 1968-69 evaluation differed from the two 
previous evaluations in two important respects. First, the 
South Carolina Center was allowed and encouraged (as were other 
centers) to participate actively in the design of evaluation 
procedures including intervention techniques for use in its 
assigned region. Although common national data again were col- 
lected by each of the centers across the country, each center 
was also free to test the effectiveness of selected and/or con- 
structed intervention techniques subject, of course, to national 
approval. In effect, a strong experimental dimension was added 
to the generally normative nature of the previous evaluations. 

The second respect in which the evaluation work of the 
South Carolina Center differed during 1968-69 was in the ex- 
istence of cooperation and continued communication with two 
other centers following the same evaluation design. The Uni- 
versity of Texas and Tulane University had worked with the 
University of South Carolina in the development of the eval- 
uation procedures, and they in fact replicated the investigation 
in their respective regions. 
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In summary, it may be reported that the South Carolina 
replication proceeded with a minimum of difficulties and all 
testing schedules and quality control criteria were met. The 
intervention language development program was conducted as 
planned at both the Henderson, North Carolina, and the Vero 
Beach, Florida, centers. The present document is a final re- 
port of the evaluation and includes a statement of the problem, 
the evaluation design, the evaluation procedures, the analysis 
of the data, and the investigators’ conclusions and recommendations. 
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STATEMENT OF THE PROBLEM 

The evaluation of the effectiveness of Project Head Start 
programs across the nation has been a most difficult task for 
fairly obvious reasons. Project Head Start is not monolithic in 
nature. Programs differ from locale to locale, with respect to 
many variables — the type of children served, the philosophies of 
the personnel involved, the available equipment and material, and 
the degree of parental and community involvement, to name only a 
fev;. There are certainly excellent Head Start programs and there 
are probably poor ones. Thus, a general common evaluation of a 
large number of programs is likely to produce data of indifferent 
or contradictory implications. Naturally, it has been impossible 
to rigorously control (even if this were desirable) the instruc- 
tional programs of the various centers. 

Recognizing the above to be true and welcoming the encourage- 
ment of the Office of Evaluation to concentrate on intervention 
programs in controlled situations, the staff of the Evaluation and 
Research Center at the University of South Carolina selected lan- 
guage development as a critical instructional area. Further, 
various combinations of programs, materials, and extent of teacher 
training were identified as independent variables for evaluation. 
Thus, the general problem of the evaluation was to select and/or 
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develop certain language development programs, materials, and types 
of teacher training to accompany these materials ; and then to com- 
pare combinations of these in a controlled investigation. 
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THE EVALUATION DESIGN - A SUMMARY 

Rationale: The Prominent Variables 

Inasmuch as the evaluation was intended to assess the ef- 
fectiveness of programs in language development, the first step 
in the design was that of identifying those instructional vari- 
ables to be included. A basic consideration was the selection 
of "packaged" (already developed, tested, and published) pro- 
grams insofar as this was possible in order to increase the 
consistency of the instructional programs across experimental 
groups . 

The Buchanan Readiness in Language Arts' * - program almost im- 
mediately appeared to have significant relevance. The Buchanan 
program is one of the few programs available in reading readi- 
ness designed for preschool children; it is based upon what 
appears to be sound learning theory considerations and it is 
phonetically consistent in its presentations. Experiences in 
following instructions, discriminating visually, learning to 
pose appropriate questions, making predictions, and drawing ac- 
curate conclusions are all integral elements of the Buchanan 
program. Just as importantly, the child T s initial experiences 
with classroom learning are designed to be highly successful. 

■^Cynthia Dee Buchanan , Readiness in Language Arts (Palo 
Alto, Calif.: a Sullivan Associates Program from Behavioral 

Research Laboratories, 1967) . 
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The program includes basic language arts skills and concepts; 
and the special emphasis on reading and spelling provides the 
learner with a good headstart while providing a continuing series 
of successful experiences. In view of the apparent appropriate- 
ness- of the Buchanan program, the decision was made that it would_ 
be a common base in each of the experimental groups. 

The second prominent variable selected for the evaluation 
was the use of supplementary programs in combination with the 
Buchanan program. The first of these to be defined was the 
"teacher innovated" supplement suggested by the Buchanan. This 
provision allows the teacher to freely supplement the basic pro- 
gram with her own materials and techniques. No formal materials 
are supplied. The next supplement chosen was the enrichment 
materials developed and recommended by the producers of the 
Buchanan program; a teacher’s manual with extensive instructions 
is provided. The third supplement, the Swanson Supplement , was 
developed and packaged at the University of South Carolina es- 
pecially for use in the present evaluation. (See Appendix A 
for sample materials from Swanson Teacher Manuals.) 

A third variable to be evaluated was the use of a specif- 
ically defined reinforcement schedule which was developed at the 
University of South Carolina for use in the evaluation. (The 
teacher’s manual for the Reinstein Reinforcement Program is 
presented in Appendix B.) 

The final prominent variable to be tested in the investi- 
gation -was varying degrees of orientation and teacher training.. — 



O 



-7- 



in the use of the Buchanan program and of the various supplements. 
Some teachers would be given only short orientation periods; 
others would receive more extensive instruction in both the use 
of one or another of the three supplements and the Reinstein 
Reinforcement Program . 

The general evaluation design included five experimental 
groups of two classes each in each of the three regions (South- 
east, Mid-south, and Southwest) and three control classes in 
each of the regions. The general configuration of the design, 
including all three regions, is presented in Figure 1. 



Region 


Replications 


Experimental Groups 


Control Groups 


1 


2 


3 


4 


5 


Type 1 


Type 2 


Southeast 
(U. of S. 
Carolina) 


Replication I 
English Speaking 
















Replication II 
















tfid- South 

(Tulane 

Univ.) 


Replication I 
English Speaking 
















Replication II 
















Southwest 
(Univ. of 
Texas) 


Replication I 
English Speaking 
















Replication II 

















Fig. 1. — General Design for the Total Evaluation 



As indicated previously, the five experimental groups would 
use the Buchanan program with various combinations of supplements, 
teacher training, and reinforcement. These combinations are 
listed in Figure 2. 
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Experimental Group 


Supplement 


Training 


Reinforcement 


Number One 


Teacher 

Innovated 


One-half day 
Orientation 


No formal 
program 


Number Two 


Buchanan 

Supplement 


One-half day 
Orientation 


No formal 
program 


Number Three 


Swanson 

Supplement 


One-half day 
Orientation 


No formal 
program 


Number Four 


Swanson 

Supplement 


One-half day 
Orientation 
plus two 
weeks training 


No formal 
program 


Nuniber Five 


Swanson 

Supplement 


One-half day 
Orientation 
plus three 
weeks training 


Reinstein 

Reinforcement 



Fig* 2. --Treatment Combinations to be Administered Experi- 
mental Groups in Each of Three Participating Regions 



In order to test the effects of the several variables on a 
sample of Head Start children, the investigators found it necessary 
to purchase and/or construct the materials, arrange a program of 
teacher training, test the sample subjects before and after ex- 
prosure to the instructional programs, and carefully monitor the 
classes during the year to determine that the various programs 
were being implemented appropriately. 

With respect to the three control groups (Figure 1) in each 
.region, these would not use the Buchanan program, but two of the 
three (Type 1) would be monitored as would each of the experimental 
groups. Control Group Type 2 in each region would not be monitored. 
This arrangement would allow the investigators to assess the effect 
that the periodic appearance of the monitor would have on the classes. 



0 



Inasmuch as each treatment was administered to two classes 



in each region, the three Evaluation and Research Centers repli- 
cated the evaluation both within their own regions and across 
regions. Obviously, it was important that classes be as homogene- 
ous as possible across experimental and control groups in order to 
assume equality of groups. But at the same time, it was desirable 
that heterogeneity exist between the classes in each treatment 
(but not across treatments) for the purpose of observing any inter- 
action or reversal effects. In view of the fact that the inter- 
vention program was a language development program, it seemed im- 
portant that the two replications within each region differ with 
respect to the language or dialect of the subjects. That is, 
replication I in each region would be English-speaking subjects, 
but it seemed appropriate to perform replication II in each region 
with subjects of another language or dialect if at all possible. 

Instrumentation 

The instruments required to collect the data for the evaluation 
fell into three general categories. First, there were those instru- 
ments selected on the national level and representing a common core 
of measurements which would be administered to all subjects in all 
regions participating in the evaluation. These instruments included 
those to be administered before the subjects 1 Head Start experiences 
and again afterward „ as well as a group of ’'middle 11 measures +- o be 
administered between the pretesting and posttesting. Other instru- 
ments stipulated by the national guidelines were designed to solicit 
data related to biographic factors of children and staff members. 
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data related to the instructional programs, data related to the 
degree of parent participation, and other relevant data (see Appen- 
dix C for a full listing of those instruments required for use by 
all Evaluation and Research Centers) . 

The second category of instruments to be used in the investi- 
gation were those selected by the present investigators to assess 
the influence of the experimental language development programs. 

These instruments would be administered before and after the 
language instructional program. The first of these was the 
Metropolitan Readiness Tests . This series of tests is considered 
by many to be one of the best available for measuring general 
readiness. Included in the Metropolitan are items assessing 
mathematical readiness as well as reading readiness. The second 
instrument selected in this category was the Gates-MacGinitie Read - 
ing Tests - Reading Skills . The Gates-MacGinitie is oriented toward 
reading readiness and thus provided another measure of language and 
reading development. The final tesu selected for the language pro- 
gram evaluation was the Illinois Test of Psycholinguistic Abilities 
(Revised Edition) . This test concentrates on language ability and 
is the only one of its kind that appears to have satisfactory 
standardization data. 

The third category of instruments used in the evaluation were 
those developed by the University of South Carolina Evaluation and 
Research Center specifically for the quality control of the instruc- 
tional programs (experimental and control treatments) and the collec- 
tion of test data in the field. One of these was the Head Start 
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Intervention Check List (see Appendix D) , and it was used by field 
monitors to report weekly on the performance of those teachers 
participating in the evaluation. The data obtained with the instru- 
ment were utilized by the Evaluation and Research Center staff to 
make field corrections and to maintain a continuous record of each 
teacher’s performance with each experimental language program in- 
cluding two of the three control groups. 

Two other instruments were utilized to maintain a high level 

of quality in the collection of data related to seven of the 

2 

national evaluation instruments and the three language development 
and achievement tests." One of these, the Examiner Evaluation Form 
(Appendix E) was used by full-time quality control persons to evalu- 
ate the performance of data collectors. Information obtained 
through the use of the instrument was utilized to make corrections 
in testing practices as they were observed. 

The other instrument developed at the University of South 
Carolina to control the quality of data being collected in the 
field was the Report of Testing Conditions (see Appendix F) . This 
form was used to continually assess the conditions under which the 
testing of sample subjects was performed. Through use of this 

n 

These tests were: Stanfora-Binet , (long form) ; Birch Response 
Style ; Inventory of Factors Affecting the Stanford-Binet ; Gumpgookies 
WPPSI-Animal House : Revised Pre-School Inventory ; Play Situation- 
Picture Board Sociometric Technique . 

a 

■These tests were: Revised Illinois Test of Psycholinguistic 
A bilities : Metropolitan Readiness Test : and Gates-MacGinitie Reading 
Tests-Reading SKills . 



instrument, reports were filed with the Evaluation and Research 
Center on the conditions under which various tests were administered. 



Sample Selection 

Thvi selection of Head Start Centers, classes, and children 
to be included in the evaluation was based on two groups of cri- 
teria. First, the sample had to meet national sample requirements. 
These stipulations included the following: 

1. There must be 120 sample eligible children in the 
design (of each region) at the end of posttesting. 

2. The sample children must have had no previous Head 
Start experience. 

3. The sample children must be between the ages of 
three and one-half to four and one-half, or four 
and one-half to five and one-half years of age. 

4. At least sixty- five percent of each class must be 
sample eligible. 

In addition to the national requirements listed above, addi- 
tional criteria for the sample were necessitated by the design of 
the evaluation as previously described. These included: 

1. At least two different Head Start Centers must be 
utilized in the evaluation, preferably in two 
different states. Further, each center must have 
at least six classes which should qualify as sanple 
eligible . 

2. Classes within centers must be separated geographically 
to the extent that treatment "leakage" would be mini- 
mized among experimental and control classes. 

.3. Teachers in the participating centers must volunteer 
to attend a two to three week training session in 
the use of the programs at the University of South 
Carolina. 

4. Center administrative personnel must be willing to 
cooperate extensively in the evaluation. 



Operational Hypothesis and Auxiliary Questions 
The operational hypothesis to be tested in the evaluation was 
formulated as follows: 

The language achievement of the experimental and 
control classes participating in the evaluation will 
support a ranking in effectiveness of the treatment 
methods (Fig. 1 and Fig. 2) of the following order (most 
effective to least effective): Group 5, Group 4, Group 

3, Group 2, Group 1, Control Groups 1 and 2. 

In addition to the specific operational hypothesis stated above, 
the following questions were to be considered as sample and data permitte 

1. Will Head Start classes in one region score sig- 
nificantly higher on the average on any of the 
tests administered than children in another region? 

2. Will classes in which the children speak a language 
or dialect other than English differ significantly 
on any of the pretests from those classes in which 
English is spoken by the children? 

3. Will the two control groups perform differently on 
any of the tests administered (monitor versus non- 
monitor effect)? 

4. Will there be significant interaction between repli- 
cations and treatments (inconsistency of treatments 
across replications) in cases where the second repli- 
cation of a region is comprised of children speaking 
languages or dialects different from English? 

5. Will the effectiveness of the treatments differ as a 
function of the age of the children in the various 
classes? 
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EVALUATION PROCEDURES 
Programs and Supplements 

An initial task in the implementation of the evaluation was 
the completion of the Swanson Supplement and the Reinstein Rein - 
forcement Program . Major elements of the Swanson Supplement had 
been developed and field tested prior to the summer of 1968, but 
the work of constructing supplemental units to correspond to each 
of the Buchanan lessons and of packaging these had to be completed 
during the summer and early fall of 1968. The Reinstein Program , 
although less time-consuming in its development, also had to be 
completed before teacher training could begin. 

Sample Selection 

Simultaneous with the development of the instructional 
supplements and materials was the selection of the sample centers, 
teachers, and children to be included in the evaluation. The 
University of South Carolina Center began the selection process 
by sending questionnaires with cover letters (see Appendix G) 
to all full year Head Start programs in Virginia, North Carolina, 
South Carolina, Georgia, and Florida. The questionnaire was de- 
signed to obtain data which would indicate which of the centers 
would meet the sampling criteria (see page 12) . The cover letter 
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which accompanied the questionnaire gave the center directors the 
general form of the evaluation and requested them to indicate 
whether or not they would participate if selected. 

Although many centers responded and stated their eagerness 
to participate, only one center met all criteria. That center 
was the Henderson (North Carolina) Full-Year Head Start Program. 

Another center, in Vero Beach, Florida, met all criteria 
except that one of the six classes had only seven sample-eligible 
children. Permission was requested of Dr. Lois-ellin Datta to 
use Vero Beach as the second sample center with this limitation. 
Permission was granted, and Henderson and Vero Beach officially 
were selected as the 1968-69 sample centers. At the time the 
centers were selected, a total of 185 children were available for 
inclusion in the evaluation. One hundred and five of these were 
enrolled in seven classes at Henderson and eighty were enrolled 
in six classes at Vero Beach. Five classes in each location were 
selected as experimental classes; two of the classes at Henderson 
and one at Vero Beach were selected as control classes. 

Inasmuch as the evaluation was intended to assess the effec- 
tiveness of programs in language development, replication 1 in each 
region would be English-speaking subjects. To be certain a dialect 
was not present which might contaminate the findings, an investiga- 
tion was done at the Henderson, North Carolina center. Dr. Arthur 
I Weiss, Head of the Speech Pathology and Audiology Program at the 
University of South Carolina, concluded that eighteen of the 
twenty-two children tested had "mild-to-strong" Southern Negro 
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Dialects but that the dialect would not interfere with the admin- 
istration of various types of psychological and cognitive tests. 

(See Appendix H.) In neither location, did the language or dialect 
of the subjects differ from standard English to the extent that 
they could be considered as "non-English" in the second replication. 
Therefore, language difference was not a variable in the Southeast 
replications. 

Teacher Training 

The evaluation design required that all teachers of experi- 
mental classes undergo a one-half day orientation period with 
the Buchanan program. The design further required that teachers 
in experimental groups four and five and all monitors receive an 
intensive training program in the use of the materials and tech- 
niques to which they were assigned. In the case of teachers of 
experimental groups one, two, and three, the orientation sessions 
were carried out by each of the regional Evaluation and Research 
Centers independently But the training of teachers for experi- 
mental groups four and five and the monitors was done at the Uni- 
versity of South Carolina for all three regions. 

Teacher training began at the University of South Carolina 
on the morning of October 7, 1968, and continued through October 
25. During the week of October 7, the training staff emphasized 
the Buchanan program; the week of October 14, teachers were trained 
in the use of the Swanson Supplement . In the final week of train- 
ing, teachers of experimental group five and the monitors received 
instruction in the use of the Reinstein Reinforcement Program . 
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After the first day of orientation, the format for the 
training session was one of class study and immediate application 
of the methods in field situations (see Schedule of Daily Events, 
Appendix I). Teachers were assigned to one of four training 
groups for practice in two local Head Start Centers. Each of the 
Centers made two classes available for training purposes, thus 
making it possible for each of the trainees to teach once every 
other day. 

Each teaching session was approximately one-half hour in 
length. The trainee teacher was responsible on each occasion for 
presenting one complete lesson to a class of Head Start children. 
Trainees who were not teaching at the time and the monitors were 
observers in the classroom during the lesson. 

Following each teaching session (three were scheduled each 
day) , an evaluation session was held with each group under the 
direction of the University of South Carolina Center training 
staff. This session was designed to allow immediate remediation 
of improper techniques or handling of materials as well as to per- 
mit positive reinforcement of effective practice. 

Initially, teachers were directed to follow manuals very 
closely. As they became increasingly familiar with the materials, 
they were encouraged to individualize their teaching as much as 
possible while adhering to the basic objectives and content of 
the programs. During the final week of training, the monitors 
were given the Intervention Check hist (see Appendix D) and 
trained in its use. The monitors then used the instrument under 
supervision in the field training sessions. 
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A total of twenty- two teachers and eight monitors were 
trained at the University of South Carolina. A smaller number 
actually participated in the evaluation, but it was necessary 
to "overtrain" in order to insure the final selection of highly 
competent teachers for participation in the evaluation and to 
provide against teacher attrition during the year. 

Pretesting 

The Evaluation and Research Center for Project Head Start 
at the University of South Carolina began pretesting in its region 
for the 1968-69 evaluation on September 25, 1968. One hundred and 
six children in the Henderson Center and seventy- six children in 
the Vero Beach Center formed an initial sample of one hundred and 
eighty-two subjects eligible for testing. 

Except for sociometric measures, all pretests had been com- 
pleted by October 22, 1968. One hundred and seventy- nine subjects 
had been administered the Stanford- Binet . the Birch Response Style , 
and the Inventory of Factors Affecting the Stanford- Binet . The 
Gumpgookies . the Wechsler Preschool and Primary Scale of Intelli - 
gence (Animal House section), and the Revised Preschool Inventory 
had been administered to 178 subjects. The Revised 111 inois Test 
of Psvcholinguistic Abilities , the Metropolitan Readiness Tests , 
and the Gates- MacGinitie Reading Tests had been given respectively 
to 177, 176, and 170 children. The sociometric measures were com- 
pleted in the Henderson Center on November 26 and in Vero Beach on 




December 12 
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Implementation of the Intervention Procedures 
The language development intervention program was made opera- 
tional in the region of the University of South Carolina during 
the month of November, 1968. The experimental language programs 
began on November 12 in Henderson, North Carolina; and the children 
in Vero Beach, Florida, began the experimental programs on November 
18. 

The general problems encountered in the implementation of the 
program were related to the packaging and shipping of materials 
from suppliers to the University of South Carolina and from there 
to the various classes in the three regions. Shipment quickly be- 
came smooth, and the task of transporting the considerable volume 
of material from the offices of the University of South Carolina 
to classes in five states grew relatively routine. 

Interim Observations and Related Instruments 
The administration of the interim measures, the Observation 
of Substantive Curricular Input and the Post Observation Inventory , 
took place during the months of November, January, February, March 
and April. During October and November, Class Registers . Parent 
Consent Forms , and the Class Facilities and Resources Inventories 
were completed. Subsequently (by February, 1969), the Child Master 
Data Form , the Teaching Staff Characteristics , and Staff Member In - 
formation instruments were completed. 

Posttesting 

Posttesting began on May 19 in Henderson and was completed on 
June 5. In Vero Beach, posttesting began on June 9 and was completed 
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on July 11 , 1969 . Teachers in both areas were informed in advance 
of the testing schedule and space requirements. Posttesting pro- 
ceeded smoothly. These tests included seven of the national evalu- 
ation instruments 1 *' and the three language development and achieve- 
ment tests. 5 

Quality Control 

The procedures designed and inplemented by the Evaluation and 
Research Center at the University of South Carolina to insure the 
quality of the evaluation data may be considered in three separate 
categories. First, there are those procedures designed to insure 
that the data collection procedures were valid. All individuals 
administering tests for the Center were required to meet certain 
criteria established by the quality control division of the staff. 
These criteria were based upon an individual's ability to relate 
to children, to master testing procedures and materials, and to 
make sound judgments in administering and scoring tests. Examiners 
administering individual tests, whatever their previous experience, 
were trained to these criteria by staff members thoroughly familiar 
with the tests and individual testing techniques. Examiners were 
then observed by the quality control staff on each test the examiner 
was to administer. Examiners had to be passed as competent by a 

^Stanford- Binet (long form) ; Birch Response Style ; Inventory 
of Factors Affecting the Stanford- Binet : Gumpgookies : WPPS I- Animal 
House; Revised Preschool Inventory : Play Situation- - Picture Board 
Sociometric Technique . 

^Revised Illinois Test of Psycholinguist ic Abilities ; Metro - 
politan Readiness Test ; and Gates- MacGinitie Reading Tests — Reading 
Skills . 
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unanimous vote of the quality control division staff. Examiners 
with limited experience who were passed by quality control were 
observed by a clinical psychology American Psychological Asso- 
ciation dip.1 ornate for final certification. 

Any examiner not passed by quality control was either dis- 
missed or given remedial training depending upon the type and 
extent of her deficiencies. Tests administered for the purpose 
of obtaining estimates of intelligence were given only by examiners 
with extensive experience. The testing performance of examiners 
and scoring procedures were observed in the field approximately 
once every two weeks by quality control staff members. Provisions 
were made for examiners to be brought in from the field for re- 
medial training when necessary. 

During quality control observations, a rating form (see 
Appendix E) was used to evaluate examiners. Examiners had to re- 
ceive a grade of B or higher on items 1 through 6 and an A on item 
7 in order to continue gathering data. In addition, the condi- 
tions under which the tests were administered are systematically 
controlled. Quality control personnel utilized the Report on Test- 
ing Conditions (see Appendix F) to monitor these situations when 
they were observing testers in the field as described above. 

The second category' of quality control procedures were those 
designed to insure that the intervention techniques, that is, the 
various experimental and control treatments, were being conducted 
in the field as specified. (See Evaluation Design section for 
description of package programs.) Monitors were trained in the 
intervention procedures at the same time that the teachers of the 



- 22 - 



experiment al group were trained. In addition, they were trained 
to use a monitoring instrument (see Appendix D) to report instruc- 
tional procedures being carried out in classes. 

The Center at the University of South Carolina had one monitor 
in the Vero Beach area and one in Henderson. Each of these monitors . 
visited each class in her area (control and experiment) once every 
six days. The order in which the classes were visited was randomized 
(by the Evaluation and Research Center) for each six-day period so 
that the teachers might not predict the day of the visit of the 
monitor. Monitors discussed instructional practices which thev 
had observed with the teacher in the experimental groups after each 
lesson and offered su gg estions for adhering as closely as possible 
to the stipulated program. When a monitor visited a classroom, 
she remained during the entire sequence of intervention procedures. 
The monitor checklist was completed at the end of each observation. 
Each Friday the monitors mailed the completed checklists to the 
Center Director, and these were reviewed by the Center staff once 
each week for irregularities. A direct line of communication be- 
tween the Center and the monitors was maintained. If further 
action was required. Dr. George H. Lackey, Jr. was consulted. He 
supervised quality control for intervention monitoring at the Uni- 
versity of South Carolina. Any questions relative to monitors and 
their duties were directed to Dr. Lackey. The monitors themselves 
were observed in the field approximately once per month by a Center 
staff member to insure that they were performing their functions 
as specified. At times, the Center member monitored a class simul- 
taneously with the monitor. Comparisons were made of the Inter- 
vention Check List following the class meeting. 
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The final category of Quality Control procedures were those 
related to the processing of data. These included a ch:?ck of the 
adequacy of the data, such as a missing name or identification 
number, and the accuracy of scoring and of recording the data. Al- 
though these procedures may appear to be more related to office 
routine than to the evaluation, they were a vital aspect of the 
data quality control system. 

As the evaluation data arrived, they were checked by locally 
stationed examiners with some aid from temporary personnel. All 
data were run through two main procedures: (a) the data identifi- 

cation check which was executed by temporary employees and Junior 
Research Assistants; and (b) the data check for scoring and re- 
cording which was done by the field examiners. 

Upon entering the Evaluation and Research Center, the data 
were first subjected to the data identification check. While no 
record was made of identification information errors, the errors 
were corrected as they were found. The first procedure in the 
data identification was the checking of every record booklet or 
answer sheet and the accompanying Digitek coding sheet for the 
following CORRECT identification information: (1) subject name; 

(2) subject number; (3) date of test administration; (4) beginning 
time of test administration; (5) ending time of test administration; 
(6) total testing time; (7) tester identification number; and (8) 
sample identification. 

After the record booklets and coding sheets were checked, 
every coding sheet was checked for agreement of correct information 
with that which had been recorded on the booklet for each particular 



administration. The necessary information for each coding sheet 
was: (1) subject number; (2) card number; and (3) tester number. 

A second general data processing procedure, the data check 
for scoring and recording, went into operation after the data identi 
fication check had been completed. At that time, every fifth test 
or answer sheet and accompanying Digitek coding sheet for each 
test was checked for correct record booklet SCORING AND RECORDING 
of information onto the coding sheets, item by item. 

In the case of an examiner being assigned the task of check- 
ing the data for scoring and recording of a particular group of 
tests for which the fifth test happened to be the one she had 
administered herself, she went to the fourth or sixth sequence 
of papers, or until there was one she had not administered, and 
continued with the every-fifth sequence in all other cases. 

All data were filed by test within the quality control data 
check division. The every-fifth test, or fourth or sixth as 
mentioned above, was pulled from the group and was marked with a 
"Q n in red in the upper left-hand corner of the front of the 
record booklet and on the back of all of the accompanying coding 
sheets in the upper right-hand comer. Examiners checked the 
record booklets and coding sheets as if they had never been 
scored or recorded, item by item. All corrections were made by 
the testers as they found errors in the scoring and recording. 

When a tester completed the data check procedures for scoring 
and recording of all test administrations of an individual test, 
she gave to quality control all the Quality Control Data Check 
Records in subject order stapled to a file. 
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The Quality Control Data Check Record was designed for use 
in connection with scoring and recording errors. The form is 
self-explanatory and had to be completed in detail on each test 
administration, including record booklet and accompanying coding 
sheets, on which was found ANY error of a scoring or recording 
nature. This form has been reproduced below. 

QUALITY CONTROL DATA CHECK RECORD 

Sample Subject Number 

Subject Name 

Test 

Date of Administration Tester Number 

No. of Scoring Errors No. of Recording Errors 

Comments (Specify Nature of Each Scoring Error) 



Additional Comments: 



Checker No. Date 

After the quality control staff reviewed and was satisfied 
with the Quality Control Data Check Records submitted by the in- 
dividual examiners for individual test groups, it advised data 
analysis that the particular test groups were ready to go to data 
analysis. From there the data went through procedures appropriate 
for preparing them for the computer center. 
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In February, 1969, Dr. Gertrude Justison made the following 
observations, following her visit to the Evaluation and Research 
Center at the University of South Carolina: 

"...are four general impressions which are worthy of 
note, if not commendation. 

"3/ The degree of systematic quality control (is3 

■ evident in all research activities from training 
to data collection and analysis. 

"2/ The organization and efficiency of materials pro- 
duction operations and distribution within rather 
serious time pressures and space limitations.... 

[is worthy of noteTJ 

"3/ The easy but responsible communication between 

staff and the careful ordered nature of articula- 
tion between separate but related functions in the 
overall research effort.... [is worthy of note^ 

"4/ The systematic, detailed recording procedures, is 
an example." 

Administrative Problems and Solutions 
At the outset of the project, a problem developed with refer- 
ence to the mode of shipping materials to the centers. When 
shipped by truck or train to Vero Beach, materials would sit in a 
warehouse miles from their destination while Vero Beach personnel 
would patiently await their delivery. It became necessary to air 
freight all materials to Vero Beach and send them a copy of the 
bill of lading so they could make inquiries if the materials did 
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not arrive when expected. With this method, few further problems 
were encountered. 

In late January, 1969, it was learned that funding for the 
Henderson region was such that classes would be unable to operate 
past the end of April. The intervention program and posttesting 
required that classes continue open through the month of May. 

Through communications between Dr. Lois-ellin Datta of the Office 
of Economic Opportunity, additional funds were provided through 
Head Start to keep classes operating through May. 

In the spring of 1969, the Regional Training Office located 
at Florida State University, Tallahassee, selected two Head Start 
teachers from Vero Beach to attend an eight- week Leadership Train- 
ing Course, not realizing they were a part of a research project. 
Communications were improved, resulting in a meeting with the 
Regional Training officers and representatives from the Evaluation 
and Research Center in late March. Objectives of the intervention 
program, along with details of the total project, were described. 
Among those attending were Dr. Tricia Godshall of the University 
of Miami, Director of the Regional Training Office in South Florida; 
Mrs. Georgia Henry, and Mrs. Carol Seefeldt, Regional Training 
officer from Florida State University. This meeting alleviated 
further misunderstanding. . 
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PART FIVE 



ANALYSIS OF THE DATA 
Introduction 

The purpose of the present evaluation was to assess the in- 
fluence of five language development programs on the learning of 
children in year-round Head Start programs. The general rationale 
and design of the investigation, including program variables and 
definition of experimental and control classes, have been presented 
in Part Three. The overall operational hypothesis to be tested in 
the evaluation was stipulated on page 13 and is repeated here for 
the convenience of the reader: 

The language achievement of the experimental and control 
classes participating in the evaluation will support a 
ranking in effectiveness of the treatment methods of the 
following order (most effective to least effective) : 

Group 5, Group 4, Group 3, Group 2, Group 1, Control 
Groups 1 and 2. 

As reported earlier, three instruments were selected to measure 
the effectiveness of the experimental language programs. These three 
were the Metropolitan Readiness Tests , the Gates-MacGinitie Reading 
Tests - Reading Skills , and the Illinois Test of Psycholinguistic 
Abilities (Revised Edition) . The results obtained in testing the 
hypothesis through the use of the three instruments are reported 
individually after a description of the sample in terms of race, 
sex, chronological age, mental age, I.Q. , and a statement of the 
general analytic strategy. 
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The Sample 

The evaluation was conducted in Henderson, North Carolina, 
and in Vero Beach, Florida. Each of the five experimental pro- 
grams was used in each location, the only difference being that 
Henderson had a second control group that was not monitored. De- 
scriptive information on race, sex, chronological age, mental age, 
and I.Q. for the experimental and control groups in each location, 
both before and after the language development programs had been 
administered, are presented in Tables 1 and 2. 

The reader will note that in Henderson the sample was com- 
posed almost entirely of Negro children. In Vero Beach, more 
than two-thirds of the children were Negro, the others being white. 
Girls outnumbered boys in both locations. Although there were 
initial differences in mean I.Q. for the two Head Start Centers, 
measures taken after the intervention program reveal mean I.Q. T s 
to be very similar. Also, the mean chronological ages of the two 
groups are very nearly the same. Only students who were available 
for both pretesting and posttesting were included in the evaluation. 
Attrition, however, was slight. 



General Strategy of the Analysis 
As indicated earlier, the three tests of language development 
used to measure the effectiveness of the intervention programs 
were administered prior to intervention and after intervention. 
The general strategy for the evaluation analysis consisted of 
analysis of variance procedures performed on the data obtained 
from the post administration of the three tests. The pre-test 
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TABLE 1 



Race, Sex, Chronological Age, Mental Age, and I.Q. 
Data for Experimental and Control Groups 
Prior to Language Development Programs 



Experimental 


Race 


Sex 


Total 


Mean 

Chronological 
Age (Months) 


Mean 

Mental Age 
(Months) 


Mean 


Groups 


N 


W 


M 


F 


I.Q. 


Treatment #1 
Henderson 


15 


0 


8 


7 


15 


62.40 


51.27 


80.87 


Treatment #1 
Vero Beach 


11 


3 


6 


8 


14 


61.43 


53.86 


85.50 


Treatment #2 
Henderson 


14 


0 


7 


7 


14 


61.43 


51.36 


81.86 


Treatment #2 
Vero Beach 


9 


4 


7 


6 


13 


62.00 


53.77 


86.15 


Treatment #3 
Henderson 


14 


0 


3 


11 


14 


62.14 


50.79 


78.64 


Treatment #3 
Vero Beach 


10 


4 


4 


10 


14 


61.64 


54.71 


87.57 


Treatment #4 
Henderson 


15 


0 


10 


5 


15 


63.27 


50.60 


77.60 


Treatment #4 
Vero Beach 


14 


1 


8 


7 


15 


63.53 


56.13 


87.20 


Treatment #5 
Henderson 


13 


1 


6 


8 


14 


63.43 


56.57 


87.79 


Treatment #5 
Vero Beach 


6 


7 


5 


8 


13 


62.85 


54.31 


84.69 


Monitored 

Control 

Henderson 


16 


0 


5 


11 


16 


63.00 


54.63 


84.25 


Monitored 
Control 
Vero Beach 


3 


4 


2 


5 


7 ■ 


63.00 


52.43 


83.14 


Unmonitored 

Control 

Henderson 


15 


0 


9 


6 


15 


63.93 


52.40 


80.00 


Totals 

Henderson 


102 


1 


48 


55 


103 


62.82 


52.52 


81.56 


Totals 
Vero Beach 


53 


23 


32 


44 


76 


62.37 


54.39 


85.97 
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TABLE 2 



Race, Sex, Chronological Age, Mental Age, and I.Q. 
Data for Experimental and Control Groups 
After Language Development Programs 



Experimental 

Groups 


Ra 


ce 


Sex 




Mean 

Chronological 
Age ( Months') 


Men1:!i n Age 

(Months) 


Mean 


N 


w 


M 


F 


Total 


I.Q. 


Treatment #1 
Henderson 


14 


0 


7 


7 


14 


70.86 


63.64 


88.86 


Treatment #1 
Vero Beach 


9 


2 


6 


5 


11 


70.00 


64.91 


92.64 


Treatment #2 
Henderson 


15 


0 


7 


8* 


15 


69.00 


64.60 


93.11 


Treatment #2 
Vero Beach 


9 


2 


5 


6 


11 


69.64 


62.18 


87.91 


Treatment #3 
Henderson 


14 


0 


3 


11 


14 


69.64 


62.29 


88.36 


Treatment #3 
Vero Beach 


10 


4 


4 


10 


14 


70.00 


66.93 


95.43 


Treatment #4 
Henderson 


13 


0 


8 


5 


13 


71.54 


63.23 


87.15 


Treatment #4 
Vero Beach 


11 


1 


6 


6 


12 


72.67 


64.42 


87.58 


Treatment . #5 
Henderson 


13 


1 


6 


8 


14 


71.36 


65.50 


90.93 


Treatment #5 
Vero Beach 


6 


5 


4 


7 


11 


71.64 


66.64 


92.18 


Monitored 
Control 
Henders on 


15 


0 


5 


10 


15 


70.80 


65.93 


92.53 


Monitored 
Control 
Vero Beach 


3 


3 


2 


4 


6 


71.17 


59.33 


81.17 


Unmonitored 

Control 

Henderson 


15 


0 


9 


6 


15 


71.73 


63.73 


88.87 


Totals 

Henderson 


99 


1 


45 


55 


100 


70.69 


64.16 


90.05 


Totals 
Vero Beach 


48 


17 


27 


38 


65 


70.82 


64.57 


90.37 



*One subject was not available for pretesting and, therefore, was 
not included in analysis. 
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scores on the language tests, as well as I.Q. and age, were pre- 
sumed to be potential covariates in cases where analysis of the 
pretest data suggested that an analysis of covariance of posttest 
data might be useful. In other words, an analysis of variance 
of pretest scores would be performed prior to the analysis of 
the posttest data. If the analysis established the equivalence 
of groups before treatments were applied, then analysis of variance 
alone would be performed on posttest data. If, on the other hand, 
evidence of initial non-equivalence of groups appeared, consider- 
ation would be given to following the analysis of variance of 
posttest data with an anslysis of covariance. 

This sequence was considered advantageous because of the 
additional assumptions required for utilization of the analysis 
of covariance. (Elashoff has recently restated in a particularly 
readable form for educators the assumptions associated with the 
use of the analysis of covariance.®) In cases where there is 
little reason to believe that the gains in precision would be sub- 
stantial through the use of analysis of covariance, it appears 
wise to depend on the analysis of variance procedures in making 
inferences regarding treatment differences. 

As each of the three tests consisted of several subtests, the 
means obtained from the tests were compared initially by a multi- 
variate test of significant differences. Subsidiary univariate 

®Janet Elashoff, ’’Analysis of Covariance: A Delicate Instru- 

ment, ’ ’American Educational Research Journal , Vol. VI, 1969, pp. 
383-401. 
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tests were performed when indicated by the multivariate test. In 
configuration, the designs for the Metropolitan Readiness Tests 
data and the Gates-MacGinities Reading Tests data were three way: 
treatments x replications x mode of test administration. Mode of 
test administration was a factor introduced in the posttesting 
situation where each treatment group was divided and one half ad- 
ministered the test individually and the other half administered 
the test in groups as stiuplated by the publishers. ^ The Illinois 
Test of Psychol inguis tic Abilities was administered to all children 
individually, and the analysis of the data obtained from it was 
therefore a two way design: treatments x replications. 

Throughout the analysis the unmonitored control group at 
Henderson (the "extra" or thirteenth group) was deleted from the 
general analysis in order that the designs be balanced with respect 
to treatments. Possible differences between monitored versus un- 
monitored groups were tested in auxiliary analyses. 

The computer program used for the majority of the analyses 
was the M&NOVA of the Biometric Laboratory, University of Miami. 
Certain analyses of covariance were performed using the Miami 
ANCOVA program. Pairwise comparisons were obtained through the 
use of Winer's suggested modification of Tukey's test (HSD)®. 

^The present investigators were interested in whether dif- 
ferential performance would be observed. Future data gathering costs 
could be reduced if the data supported the null hypothesis. 

®B. J. Winer, Statistical Principles in Experimental Design 
(New York: McGraw-Hill, 1962), pp. 101-103. 
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Metropolitan Readiness Tests 

The data obtained from the initial administration of the 
Metropolitan Readiness Tests are summarized by groups and subtests 
in Table 3 for Henderson and in Table 4- for Vero Beach. Consistent 
with the general strategy outlined in the preceding section, these 
data were subjected to a multivariate analysis of variance. A 
groups x replication interaction was observed; F = 2.34- with 30 
and 450 degrees of freedom, significant above the .001 level. A 
difference in replications also was observed but this could have 
been expected and was not an issue at this point in the analysis. 

Examinations of the discriminant weights and univariate F 
tests suggested that the observed groups x replication interaction 
was due chiefly to differential performance on the third sub-test 
of the Metropolitan Readiness Tests , Matching. The reader will 
note that Experimental Group 1 in Henderson (Table 3) and Experi- 
mental Group 3 in Vero Beach (Table 4) performed very poorly on 
this subtest when compared to the other classes in the evaluation. 

In general the results of the analysis of variance confirmed 
the hypothesis that the groups at Henderson were very much alike, 
as were those at Vero Beach, in their initial performance on the 
Metropolitan Readiness Tests . There were, however, differences 
between the replications. 

The first step in the analysis of the posttest scores was a 
multivariate analysis of variance: groups x replications x mode 

of test administration (individual or group) . The cell means of 
these scores are presented in Table 5 for Henderson and Table 6 
for Vero^Heach. 
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Initial Metropolitan Readiness Tests Data by 
Subtest's for Groups in Henderson Replication 
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Note: Inconsistencies in frequencies between the above table and Tables 1 and 2 occur because 

only children receiving both pretest and posttest were considered in the analysis. 
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Note: Inconsistencies in frequencies between the above table and Tables 1 and 2 occur because 

only children receiving both pretest and posttest were considered in the analysis. 
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Inspection of the multivariate analysis results indicated a 
significant groups x replication interaction (F = 1*52 with 30 
and 450 degrees of freedom, significant above the .041 level). 
Further analysis of this groups x replication interaction revealed 
that it was being caused by differential performance of the 
groups on the fourth subtest, Alphabet (F = 4.93 with 5 and 117 
degrees of freedom, significant above .001 level). Apparently, 
the interaction observed was chiefly a reflection of the poor 
performance of the Vero Beach Experimental Group 2. The group 
means for this subtest are presented in Table 7 by experimental 
and control groups and by replications. 

TABLE 7 

Means for Metropolitan Readiness Tests , Subtest 4, 

Alphabet, by Experimental and Control Groups 
and by Replications 



Experimental and 


Replications 


Control Groups 


Henderson 


Vero Beach 


Experimental Group 1 


11.4 


13.4 


Experimental Group 2 


13.4 


6.5 


Experimental Group 3 


13.6 


13.3 


Experimental Group 4 


11.0 


8.3 


Experimental Group 5 


13.1 


11.5 


Control Group 1 
(Monitored) 


5.7 


7.2 
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In addition to the significant groups x replication inter- 
action effect, the multivariate analysis also indicated signifi- 
cant differences on this subtest (Alphabet) among groups in the 
two replications. Therefore, it was reasonable to consider the 
simple effects within each replication. 

In the case of the Henderson replication, inclusion of the 
Control Group 2 (unmonitored) with a mean of 8.0 led to the fol- 
lowing rank ordering of the groups at Henderson with respect to 
Subtest 4, Alphabet: the Control Group 1 (monitored) had a lower 

mean than all others except the Control Group 2 (unmonitored) ; 
the Control Group 2 (unmonitored) was not significantly different 
from Experimental Groups 2, 3, and 5. This rank ordering may be 
clarified somewhat with a diagram where a continuous line drawn 
under two or more groups indicates no significant difference in 
means. The relationships among the means of the seven groups in 
the Henderson replication are presented in this fashion below: 



. . * Higher 



Means 



Lower > 



Experi- Experi- Experi- Experi- Experi- Control 2 Control 1 

mental mental mental mental mental (Unmoni- (Moni- 

32 514 tored) tored) 



Fig. 3. — Rank-Ordering of Means for Henderson Groups for 
Metropolitan Subtest 4, Alphabet. 
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Thus, it can be seen that Control Group 1 (monitored) is below 
all others except Control Group 2 (unmonitored) , and the unmoni- 
tored control is below Experimental Groups 3, 2, and 5. 

If the same type of diagram is used to present results from 
the Vero Beach replication, the reader will see that Control Group 
1 and Experimental Groups 2 and 4 are significantly less than Ex- 
perimental Groups 1 and 3 and Experimental Group 2 is also less 
than Experimental Group 5. See Figure 4 below. 



< — 


Higher 






> 


Experi- 


Experi- 


Experi- Experi- 


Control 


Experi- 


mental 


mental 


mental mental 


Group 


mental 


1 


3 


5 4 


1 


2 





Fig. 4. — Rank-Ordering of Means for Vero Beach Groups 
for Metropolitan Subtest 4, Alphabet. 



Subtest 4, Alphabet, was the only subtest of the Metropolitan 
on which significant differences in treatments were observed. 

The multivariate analysis also revealed a significant dif- 
ference in mode of test administration (F = 2.31 with 6 and 112 
degrees of freedom, significant above .038). This effect was ac- 
counted for by the first Metropolitan subtest. Word Meaning. The 
univariate analysis of this variable revealed an F value of 7.68 
with 1 and 117 degrees of freedom, significant above the .006 level. 
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In effect, the individually tested children consistently outper- 
formed the group tested children on this first subtest of the 
Metropolitan . (See Table 8 for numbers of subjects receiving 
tests through individual or group administrations in each repli- 
cation.) Inspection of the means indicated that this had hap- 
pened with four of the six groups in the Henderson replication 
and with five of the six groups in the Vero Beach replication. One 
might speculate that the finding is reflective of some systematic 
effect that operates initially in the testing situation and tends 
to disappear for later subtests. The differences observed between 
the two replications on the pretest administration of the Metro - 
politan were not observed in the analysis of the posttest data. 

TABLE 8 



Numbers of Subjects Receiving Metropolitan Readiness 
Tests through Individual or Group Administration by 
Experimental or Control Groups and Replications 



Experimental and 


Henderson 


Vero 


Beach 


Control Groups 


Individual 


Group 


Individual 


Group 


Experimental 1 


7 


7 


5 


5 


Experimental 2 


9 


S 


S 


6 


Experimental 3 


7 


6 


6 


6 


Experimental 4 


5 


6 


6 


6 


Experimental 5 


6 


7 


5 


S 


Control Group 1 
(Monitored) 


10 


5 


2 


4 


Control Group 2 
(Unmonitored) 


9 


6 
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At this point in the analysis, the decision was made to pro- 
ceed with analysis of covariance procedures using various predictor 
sets. Therefore, the data obtained from the posttest administra- 
tion of each of the Metropolitan Readiness Tests subtests were 
analyzed using first the corresponding pretest, then the correr 
sponding pretest with pretest I.Q. scores, and finally the pretest, 
the I.Q., and the age of the subjects as covariates. The results 
of these analyses served only to verify those findings previously 
presented, namely: (1) with respect to Subtest 4, Alphabet, there 

was a significant groups x replication interaction; differences 
were found in the groups within replications but patterns were 
not discernible, and (2) individual testing proved to be superior 
to group testing in the case of the first Metropolitan subtest. 

Word Meaning. 

The performance of the analysis of covariance did provide 
evidence, however, as to the usefullness of the procedure in the 
present circumstance. These items may be enumerated as follows: 

1. The correlations between pretest and posttest perfor- 
mance of students on the first three subtests of the Metropolitan 
(Word Meaning, Listening, and Matching) were very small. Linear 
regression on the pretest data was not reasonable; these were of 
no practical use as predictors. • 

2. The pretest I.Q. proved to be a significant predictor, 
except in the case of Suhtest 1, Word Meaning. 

3. The age of subjects was of little or no value as a 




predictor. 
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4. In general, the precision gained through the use of the 
analysis of covariance was quite small. 

No comparisons of the monitored and unmonitored control groups 
in the Henderson replications have been presented in the preceding 
analysis. The posttest means for each of the subtests for the two 
groups are presented with error terms and t values in Table 9 below. 
The reader will note that the two groups differed significantly on 
only Subtest 6, Copying. 

TABLE 9 



Comparisons of Henderson Monitored and Unmonitored Control 
Groups by Metropolitan Readiness Tests Subtests 



Metropolitan 

Subtest 


Control 1 
(Monitored) 
(N = 15) ' 


Control 2 
(Unmonitored) 
(N = 15) 


Mean 

Square 

Error 


t 

Value 


1 - Word Mean- 
ing 


6.1 


5.8 


2.76 


0.50 


2 - Listening 


9.4 


8.5 


4.39 


1.18 


3 - Matching 


6.5 


5.6 


4.31 


1.19 


4 - Alphabet 


5.7 


8.0 


12.28 


-1.80 


5 - Numbers 


7.9 


8.1 


5.49 


0.23 


6 - Copying 


2.3 


3.7 


3.54 


2.04* 



*Significant above the .05 level, two tailed test. 



The Table of F ratios for the three way analysis of 
the Metropolitan subtest scores has not been presented in the 
text of the present report. These statistics for both the pretest 
and the posttest analyses are presented in Appendix J. Intcr- 
correlations among subtests, for both pretest and posttest data, 
appear in Appendix K. 
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Gates-MacGinitie Reading Tests 

The treatment of the data obtained through the two adminis- 
trations of the Gates-MacGiniiie Reading Tests-Readlng Skills was 
performed in much the same manner as was the analysis of the 
Metropolitan data described in the preceding section. Again, the 
first step consisted of a multivariate analysis of the pretest 
scores for significant differences. The ciata obtained from the 
pretest administration are summai' , ized by experimental and control 
groups and subtests in Table 10 for Henderson and in Table 11 for 
Vero Beach. 

The only significant effect indicated by the multivariate 
analysis of the pretest scores was a difference in replications. 
Inspection of the groups across replications revealed that the dif- 
ference was being caused by the Vero Beach students scoring sig- 
nificantly higher than the Henderson students on Subtest 1, Listen- 
ing Comprehension; Subtest 2, Auditory Discrimination; and Subtest 
3, Visual Discrimination. No significant differences among experi- 
mental and control groups within replications were found, however; 
and it was reasonable to conclude that an analysis of variance of 
posttest scores was appropriate. The means of these posttest scores 
by experimental and control groups and by subtests of the Gates - 
MacGinitie are presented in Table 12 for the Henderson replication 
and in Table 13 for the Vero Beach replication. 

. The multivariate analysis of the posttest scores indicated a 
difference in mode of test administration that approached significance 
(F - 1.96 with 8 and 105 degrees of freedom, significant above the 
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Initial Gates-MacGinitie Reading Tests-Reading Skills Data 
by Subtests for Groups in Henderson Replication 



46 



ij 



O 

ERLC 




Note: Inconsistencies in frequencies between the above table and Tables 1 and 2 occur because 

only children receiving both pretest and posttest were considered in the analysis. 
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Note: Inconsistencies in frequencies between the above table and Tables 1 and 2 occur because 

only children receiving both pretest and posttest were considered in the analysis. 
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TABLE 13 

Means Per Cell of Gates-MacGinitie Reading Tests - Reading Skills Subtests 
Scores for Vero Beach Sample by Experimental and Control Groups 
and by Individual or Group Test Administrations 
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.059 level). As in the case of the Metropolitan data, the cause 
this ef t was found in the first subtest. Again, individually 
tested children systematically outperformed group tested children 
with the effect fading after the first subtest. The numbers of 
children receiving the test individually or in groups are presented 
by experimental and control groups and replications in Table 14. 

The multivariate analysis also indicated significant differences 
among experimental and control groups in their performance on Sub- 
test 2, Auditory Discrimination; Subtest 5, Letter Recognition; and 
Subtest 6, Visual Motor Coordination. The effects associated with 
Subtest 5 differed from those of Subtest 2 and Subtest 6 in that 
they suggested the possibility of a significant groups x replicate 
interaction. The multivariate analysis did not reveal this effect, 
but the univariate indicated an interaction that was significant 
above the .009 level. The multivariate test, of course, is used 
to minimize the probability of Type I error when performing a large 
number of tests at a given level of significance. Nevertheless, 
when a univariate result appears which may shed light on a sit- 
uation, the investigator is obliged to examine simple effects. 

These were inspected in the present case, and the means for the 
groups in each replication are presented in Table 15. 

From the table, it can be seen that the groups x replications 
interaction is probably being caused by the relatively poor per- 
formance of Vero Beach Experimental Group 2 on this subtest (Letter 
Recognition) . This result was consistent with an interaction reported 
earlier in the analysis of the Metropolitan data. It is also not 



51- 



TABLE In- 

Number's of Students Receiving Gates-MacGinitie Reading 
Tests - Reading Skills through Individual or Group 
Administration by Experimental and Control Groups 
and by Replications 



Experimental and 


Henderson 


Vero 


Beach 


Control Groups 


Individual 


Group 


Individual 


Group 


Experimental 1 


7 


6 


5 


5 


Experimental 2 


6 


7 


6 


5 


Experimental 3 


7 


5 


6 


5 


Experimental 4 


5 


6 


6 


6 


Experimental 5 


6 


7 


4 


5 


Control 1 
(Monitored) 


9 


6 


2 


4 


Control 2 
(Unmonitored) 


8 


7 







TABLE 15 

Gates-MacGinitie Subtest 5, Letter Recognition, Means 
of Experimental and Control Groups by Replications 



Experimental and 
Control Groups 


Henderson 


Vero Beach 


Experimental 1 


13.9 


14.4 


Experimental 2 


15.4 


8.9 


Experimental 3 


14.2 


13.6 


Experimental 4 


13.8 


11.3 


Experimental. 5 


14.2 


13.1 
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Control 1 



6.1 



8.2 
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surprising inasmuch as the Metropolitan Subtest 4 (where that inter- 
action occurred) is Alphabet and the Gates-MacGinitie Subtest 5 
is Letter Recognition. Vero Beach Experimental Group 2 simply was 
weak in this area and this is reflected in the two interactions. 

At this point, experimental and control groups were considered 
in each replication separately for significant differences. The 
Henderson Control Group 1 (monitored) was found to be significantly 
lower than all experimental groups, and Control Group 2 (unmonitored). 
There were no other significant differences among groups. See 
Figure 5 below. 



— Higher 



Means 



Lower -> 



Experi- Experi- Experi- Experi- Experi- Control Control 

mental mental mental mental mental 

2 3 5 1 4 2 1 



Fig. 5. — Rank-Ordering of Means for Henderson Groups on 
Gates-MacGinitie Subtest 5, Letter Recognition. 

In the Vero Beach replication. Control Group 1 (monitored) 
scored significantly below Experimental Groups 1 and 3, and Experi- 
mental Groups 2, 4 and 5 were significantly below Experimental 
Group 1. See Figure 6 below. 



< Higher Means Lower > 

Experi- Experi- Experi- Experi- Experi- Control 

mental mental mental mental mental 

1 3 5 4 2 1 



Fig. 6. — Rank-Ordering of Means for Vero Beach Groups on 
Gates-MacGinitie Subtest 5, Letter Recognition. 
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As noted earlier, the multivariate analysis also indicated 
significant differences among experimental and control groups on 
Subtest 2, Auditory Discrimination and on Subtest 6, Visual Motor 
Coordination. In the former case, that of Subtest 2, pairwise 
comparisons were performed again using Winer's modification of 
Tukey’s technique. No significant differences were observed between 
experimental and control groups. This result may occur, of course, 
in that a significant multivariate F does not insure that pairwise 
comparisons will be significant. 

With respect to the Gates-MacGinitie Subtest 6, Visual Motor 
Coordination, the pairwise comparisons did yield one significant 
difference in groups. Experimental Group 4 was found to be signif- 
icantly poorer in performance than Experimental Group 5. The means 
for the experimental and control groups by replications and total 
are presented in Table 16. 

TABLE 16 

Means for Experimental and Control Groups by Replications 
and Total for Gatesr-MacGinitie Subtest 6, 

Visual Motor Coordination 



Experimental and 
Control Groups 


Henderson 


Vero Beach 


Total 


Experimental 1 


8.5 


8.0 


8.3 


Experimental 2 


8.1 


6.9 


7.6 


Experimental 3 


9.3 


9.0 


9.2 


Experimental 4 


8.0 


4.7 


6.3 


Experimental 5 


10.9 


9.7 


10.4 


Control 1 


6.9 


9.0 


8.0 
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In addition to the procedures described on the preceding 
pages, analyses of covariance procedures were utilized with the 
Gates-MacGinitie data in the same manner as with the Metropolitan 
data described, earlier. Again predictor sets including correspond- 
ing pretest scores, pretest I.Q. scores, and age were used as co- 
variates. No new results were obtained. 

Inasmuch as mode of test administration appeared to make no 
difference after the first subtest, the design was collapsed to a 
two-way, groups x replications design and analysis of covariance 
performed. As would be expected, these results agreed with those 
obtained from the three-way analysis. The predictive value of the 
corresponding Gates-MacGinitie pretest scores and age w s small as 
in the case of the Metropolitan with the pretest I.Q. scores a 
little better for particular subtests. 

As indicated earlier, separate analyses were performed to de- 
termine significant differences in the performances of Henderson 
Control Group 1 (monitored; and Control Group 2 (unmonitored) . Two 
significant results were obtained. The unmonitored Control Group 2 
scored significantly higher than the monitored Control Group 1 on 
the Gates-MacGinitie Subtesr 5, Letter Recognition, and Subtest 6, 
Visual Motor Coordination. The posttest means for each of the sub- 
tests for the two groups are presented with error terms and t values 
in Table 17. 

The table of F ratios for the total three-way analysis of the 
Gates-MacGinitie Reading Tests subtest socres is presented in Ap- 
pendix L. Intercorrclations of Gates-MacGinitie subtests for pre- 




test and posttest administrations appear in Appendix M. 



Comparisons of* Henderson Monitored and Unmonitored Control Groups 
by Gates-MacGinitie Reading Tests Subtests Means 
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Significant above the .01 level, two tailed test. 
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Illinois Test of Psycholinguist ic Abilities 

Inasmuch as each of the twelve subtests of the Illinois Test 
of Psycho-linguistic Abilities was administered to each child on an 
individual basis, the dimension representing mode of test adminis- 
tration was not a part of the analysis. Hence, the design was a 
two way analysis (experimental and control groups x replications) 
with Control Group 2 (unmonitored) at Henderson again being con- 
sidered in a separate analysis. 

A multivariate analysis performed with the pretest data (see 
summary of data for Henderson in Table 18 and for Vero Beach in 
Table 19) from the subtests of the IJ-Iinols Test of Ps yc hol ing - 
ulstlc Abilities yielded only one significant difference in initial 
status. As in the earlier cases, there was a significant difference 
between replications. Inspection of the data indicated that it was 
caused by siiperior performance of the Vero Beach students on Sub- 
test 3, Visual Sequential Memory, and Subtest 10, Manual Expression. 
There were no other significant multivariate effects in the analysis 
of the pretest data. 

As in the two previous cases, the analysis of the posttest 
data began with a multivariate analysis by groups and replications 
for each of the subtests. The means for the groups are presented 
in Table 20 for Henderson and in Table 21 for Vero Beach. 

No significant differences in posttest performance were obtained 
through the use of the multivariate test of significance. Two sig- 
nificant univariate results, however, suggested a replication dif- 
ference on the Illinois Test of Psychol inguistic Abilities , Subtest 2, 



O 
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Visual Reception, and Subtest 12, Sound Blending. Ihe reader will 
note that a comparison of Tables 18 and 19 for these two subtests 
revea] s a tendency for Vero Beach students to outperform Henderson 
students . 

The multivariate analysis of the data was followed (as in the 
two previous analyses) by an analysis of covariance using: (1) 

corresponding pretest scores, (2) corresponding pretest scores and 
pretest I.Q. scores, and (3) corresponding pretest scores, pretest 
I.Q. scores, and age, as predictor sets. No additional information 
was revealed. 

Finally, the Control Group 1 (monitored) and the Control Group 
2 (unmonitored) at Henderson were compared for significant differences. 
There were no significant differences in the two groups on any of 
the twelve subtests. 

•ft ■ 

The table of 1 ? ratios for the total two way multivariate ^anal- 
ysis of the Illinois Test of Psycholinguistic Abilities subtests 
appears in Appendix N. Intercorrelations of the subtest for pre- 
test and posttest administrations are presented in Appendix 0. 
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Posttest Groups Means for Subtests of the Illinois Test of Psycholinguistic 
Abilities for Henderson Sample by Experimental and Control Groups 
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Posttest Groups Means for Subtests of the Illinois Test of Psycholinguist ic 
Abilities for Vero Beach Sample by Experimental and Control Groups 
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Significant Differences in I.Q. Test Scores 

A final aspect of the evaluation data analysis involved the 
scores obtained from the pretest and posttest administrations of 
the Stan ford-Binet Intelligence Scale . Although the Vero Beach 
students had apparently scored somewhat higher than the Henderson 
students on the pretest, there seemed to be little differences in 
the posttest scores. 

Therefore, a multivariate analysis was performed on the pre- 
test I.Q. scores for the two replications. A significant dif- 
ference in replications (F = 4.10 with 1 and 130 degrees of free- 
dom, significant above the .045 level) was indicated in favor of 
the Vero Beach students . The same analysis was performed using 
the posttest I.Q. scores and no significant differences between 
replications appeared. In other words, initial differences in I.Q. 
scores between Henderson and Vero Beach children had disappeared 
after Head Start intervention. 

At this point, an analysis was performed to determine which 
classes in each replication had increased their posttest mean I.Q. 
scores significantly over their pretest I.Q. scores. This analysis 
consisted of performing t tests for correlated means on the pretest 
and posttest scores for each group. The pretest and posttest means, 
differences in pretest and posttest means, and t values for each 
group in each of the two replications are presented in Table 22. 

The reader will note that the groups in the Hendersrn rep- 
lication increased their means scores in all cases, ranging from 
as little as four points to as high as twelve points. All were 




TABLE 22 



Pretest and Posttest Means, Differences, and t Values Based 
on T.Q. Scores by Experimental and Control 
Groups and Replications 



Repl ic ation Experimental 
and Control 
Groups 


Pretest 

Means 


Differ'ences 
Posttest in Pretest 

Means Posttest 

Means 


t 

Values 


Henderson Exp . 1 


81.4 


88.9 


7.5 


2.71** 


Exp. 2 


80.8 


92.5 


11.7 


4.08** 


Exp. 3 


78.4 


88.1 


9.7 


3.38** 


Exp. 4 


77.0 


87.3 


10.3 


3 . 44** 


Exp. 5 


88.3 


92.3 


4.0 


2.09* 


Control 1 


84.2 


92.5 


8.3 


3.11** 


Vero Beach Exp. 1 


84.6 


94.0 


9.4 


2.87** 


Exp. 2 


85.5 


87.9 


2.4 


0.77 


Exp. 3 


88.3 


95.4 


7.1 


2.47* 


Exp. 4 


86.9 


87.6 


0.7 


0.23 


Exp. 5 


85.5 


94.1 


8.6 


2.62* 


Control 1 


81.3 


81.2 


-0.1 


-0.02 


* Significant above 
** Significant above 


the .05 level 
the . 01 level 


, one 
, one 


tailed test, 
tailed test. 
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significant. In the Vero Beach replication, three groups improved 
their posttest performance significantly with increases in means 
of 9.4, 8.6, and 7.1. Increases in the Henderson groups were, 
therefore, greater than in the Vero Beach groups, explaining the 
differences in effects noted between the pretest multivariate 
analysis and the posttest analysis. 




PART SIX 



CONCLUSIONS 

The present evaluation was designed and conducted to test the 
effectiveness of five experimental treatments on the language de- 
velopment of a sample of Head Start children. The specific opera- 
tional hypothesis to be tested was: 

The language achievement of the experimental and control 
classes participating in the evaluation will support a 
ranking in effectiveness of the treatment methods of the 
following order (most effective to least effective) : 

Group 5, Group 4, Group 3, Group 2, Group 1, Control 
Groups 1 and 2. 

Although significant differences in experimental and control 
groups were found in the cases of several subtests, the data do 
not support the hypothesis. The language achievement of the groups 
did not rank in the order predicted. The only exception to this 
conclusion is the fact that experimental groups, in general , out- 
performed control groups on the Metropolitan Subtest 4, Alphabet 
and on the Gates-MacGinitie Subtest 5, Letter Recognition. Ap- 
parently, the Buchanan Readiness in Language Arts program, which 
served as a base for each of the five experimental treatments, 
brought about a fairly consistent change in the variable represented 
by Alphabet and Letter Recognition. This was the only pattern re- 
lated to treatment ordering found in the evaluation. 
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In the designing of the evaluation* the postion was taken 
that the treatments were different enough from each other and 
certainly from the control treatments that some general ordering, 
logically, could be expected. Two alternatives to the proposition 
that they truly were not differential suggest themselves. First, 
the criterion tests may not have been discriminating enough to 
reveal consistent differences. Inspection of the data suggests 
that this was not true. The present investigators have concluded 
that the tests used were appropriate for the evaluation. 

A second alternative, is the possibility that, although 
teachers were monitored and teacher training was an experimental 
variable, the individual effectiveness of the various teachers was 
so great that it obfuscated treatment differences if they existed 
in fact. That is to say, the personal variable was so influential 
that it masked other effects. This contention, of course, has 
been suggested in other experimental situations by other investi- 
gators and the present writers believe that it occurred in the 
Head Start evaluation reported here. It may well be that, if no 
means can be found to significantly lower teacher variability, 
experiments of this type can be expected to be of limited value 
in improving the educational process . 

In addition to the operational hypothesis, five questions 
were posed and were to be considered as the sample and the data 
permitted. These questions are restated below with the comments 
of the investigators. 
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1. Will Head Start classes in one region score 

significantly higher on the average on any 
of the tests administered than children in 
another region? ' . 

This question can be answered only through pooling the data 
from the Tulane University and the University of Texas evaluations 
and analyzing across regions. 

2. Will classes in which the children speak a 
language or dialect other than English differ 
significantly on any of the pretests from 
those classes in which English is spoken by 
the children? 

The language of the children in the two replications could 
not be considered to differ from English although there was evi- 
dence of a "mild-to-strong" Southern Negro dialect in the Henderson 
groups. Further analysis of the data from all three regions may 
serve to shed light on whether or not this dialect, after all, 
was significant in the observed effectiveness of the treatments. 

3. Will the two control groups perform dif- 
ferently on any of the tests administered 
(monitor versus non-monitor effect) ? 

The analysis of the data presqn :ed in the preceding section 
has shown that in a few cases the unnonitored control group out- 
performed the monitored control groups. The present investigators 
believe that these differences were more related to differences 
in teacher effectiveness than to the influence of the monitor. 

4. Will there be significant interaction between 
replications and treatments (inconsistency of 
treatments across replications) in cases where 
the second replication of a region is comprised 
of children speaking languages or dialects 
different from English? 
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There were no implication by treatment interactions that 
could be related to differences in language or dialect. 

5. Will the effectiveness of the treatments 
differ as a function of the age of the 
children in the various classes? 

The ages of the children in the evaluation did not vary suf- 
ficiently for this question to be answered. 

A final effect to be noted was the substantial increase in 
posttest I.Q. scores over pretest scores for the groups in the 
Henderson replication. All groups made statistically significant 
gains. In Vero Beach three of six groups increased these scores 
significantly. Clearly, the Henderson groups, both experimental 
and control, made greater progress. Was the Head Start program 
more effective generally in Henderson than in Vero Beach? Could 
this have been caused by the rural (and often geographically 
isolated) background of the children in Henderson as compared 
with Vero Beach? These are questions for which the evaluation 
provides provocative speculations but no conclusions. 
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REINSTEIN REINFORCEMENT PROGRAM 

m 

I . General Objectives of. Reinforcement Program 

The reinforcement program described herein has three basic 
objectives. First, the reinforcement program was developed to 
emphasize and strengthen the various concepts and skills introduced 
••• through the Buchanan Readiness in Language Arts materials. Such 
concepts as left and right, up and down, top and bottom, under 
and over, first and last, front and back, colors, letters, and 
words, will be constantly reviewed and re-emphasized by the rein- 
forcement materials. The rationale underlying the reinforcement 
program is that .the children must be able to understand, appj.y 
and retain the Information presented to them by means of the 
Buchanan package. The reinforcement pictures, formally called 
"progress checks", will be an indication of the- extent to which 
the lessons have been learned by the Head Start children. 

Second, the reinforcement program will attempt to instill 
within the Head Start children, the attitude that learning can 
indeed be an enjoyable experience if not actually fun. According 
to the program the children will be asked to do things which they 

■ 

like to do (e.g. color) and will receive prizes or rewards which 



they enjoy having (e.g. candy, toys). Thus, by means of the rein- 
forcement program, the association of learning and enjoyment (having 
a good time) will be strengthened. 

Third, the reinforcement prograiri attempts to develop in the 
• children attitudes that would be conductive to active participation 

in the learning process of the school, particularly with respect to 

ERIC 
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motivation to achieve and to such related variables as persistence 
and delay of gratification. The general procedure by which we hope 
to promote the above attitudes is to reinforce (emphasize, prai'se, 
call special attention to) childrens’ behavior which is indicative 
of or conductive to adequate school performance. Instead of 
.waiting for such behavior to occur by itself, the reinforcement- 
program introduces and calls forth the desired behavior so that it 
can be reinforced. (It need be mentioned here that all spontaneous 
desired behavior (anything relating to learning activity) is to 
be reinforced as well (i.e. verbally-praise) . 

The reinforcement when it does occur will be both a) immediate 
and concrete and b) delayed and symbolic. The rationale for this 
two-fold approach is to lead the children from the concrete relation- 
ships of their present environment to the abstract relationships they 
will face upon entering the formal school system. It should be 
emphasized here that this transition will be a very gradual one. 

The teacher wil], acquire a more thorough grasp of the intent 
of the reinforcement program if she keeps in mind five patterns of 
behavior common to the disadvantaged child which the program is 
attempting to overcome. 

1. The disadvantaged child does not receive enough reinforcement 
of his behavior. We, during, this program will attempt to reinforce 
all desirable behavior. 

2. Reinforcement for the disadvantaged child, when it does 
occur, usually comes from another child. This'' program emphasizes 
adult -administered reinforcement. 

3. Reinforcement for the disadvantaged child is usually only 
non-verbal. Our program will strongly emphasize verbal reinforcement 
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(praise) in conjunction with pats, hugs, etc. 

4. Reinforcement for the disadvantaged child is less focused 
in terms of being directed towards the adequacy of his specific 
acts. That is, his reinforcement is apt to consist of a rather 
vague, generalized approval such as, "That's a good boy”, or 
merely a smile, rather than such specific approving words as, " You 
tied your shoes just right " or" You really did a good job of helping 
me with the sweeping" . The teacher in this program is to emphasize 
the latter approach at every opportunity. 

5. Rej.nf or cement for the disadvantag'd child is more toward 
inhibiting behavior than it is toward encouraging exploratory 
activity. Teachers in the present program should attempt to rein- 
force only those behaviors which are or are leading to learning 
experiences. 



II. 





General Information Concerning the Reinforcement Program 

1. The reinforcement program is similar to the Buchanan 
and Supplement packages in that it requires a specific 

block of time. This block of time is approximately 20 minutes 

2. The reinforcement program consists of: 

a. Administering the progress check pictures 

b. Dispensing candy for successful performance 

c. Allowing the children to select their free play 
activity and toys when appropriate 

d. Staying with and guiding the free play activities 
of the children for the entire specified time 

3. The reinforcement program follows immediately upon the 
completion of the supplement materials for that particular 
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day. By no means can the reinforcement be postponed until, the 
afternoon or some other convenient time. 

4. The reinforcement program will occur once per day, in 
correspondence with one lesson per day on the Buchanan and 
Supplement material. However, verbal reinforcement (praise) 
should continue throughout the entire school day. 

5. Reinforcement candy will be provided for every teacher. 

The candy will be packaged in cellophane bags with five small 
pieces of candy per bag. Each child upon success with the 
progress check pictures is to receive one (1) bag of candy . 

6. Distribution of the candy should take place away from 
the teaching area whenever possible. 

7. - Reinforcement prizes (toys) will be provided for every 
teacher. A wide selection of toys will be available. It 
must be remembered, however, that the children are to be . 
given a choice of only four (4) prizes during any particular 
two week period . Every two weeks the selection of toys 
must be changed. 

8. Distribution of the prizes should take place away from the 
teaching area whenever possible. 

9. Each teacher should obtain a reasonably large cardboard 
box for each child in the class. This box can then be used to 
hold progress check pictures, candy, toys or similar objects. 
(These boxes should be obtained from the local area, for it 
would be impossible to mail such material without excess damage.) 

10. Each teacher must remember to record in the reinforcement 
book: 

a. The number of the progress check each child succeeds with 



b. The activity each child selects 

o. The appropriate number of successes each child has 
accumulated to date (starting over again after each 
four (4) successes for each child) 

d. The specific toy each child selects upon accumulating 
— four successes. 

. „ III . Reinforcement Record Book 

The reinforcement record book will provide an overall picture 
of the whple reinforcement program. The book consists of: 

a. General information pertaining to the project. 

b. Daily lesson record sheets 

0. Sample copies of each of the progress check pictures 
for every teson of the Buchanan package. 

A narrow yellow marker will be provided on which to list 
the nrmes of all the children in the class. This marker needs 
to be' advanced in the book after each lesson is completed. Reference 
to the record book wi.ll remove any hesitancy as to which progress 
check pictures are to be presented with which Buchanan and supple- 
ment lessons . It must be remembered that the record book is to be 
filled out everytime a lesson is presented. 

IV. * Step-bv-Step procedure for administration of Reinstein Reinforcement 
Program 

1. Before presentation of the very first Buchanan lesson, 
the teacher must explain to the children in appropriate terms (to 
their understanding) how this project is to work. That is, the 
• teacher need only explain that the children will learn and do many 
enjoyable things in the coming months, and that if each child will 
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pay attention to the ' activities presented and does a good job 

• with them (not necessarily perfect) they will receive some prizes 

later on. The teacher should re-emphasize that the prizes will 

only be given to those children who do exactly as they are told. 

♦ 

2. Immediately upon completion of the supplement activities, 

the teacher should again remind the children that if they do the 

♦ 

next activity just as they are told to they will receive candy, etc. 

3. The teacher will then present picture number three (3) — 
the most diffieult-to all the children and read' the instructions 

written on the picture. It should be mentioned here that the ' 

# 

* t * * • 

teacher may vary the instructions somewhat in order to make the 
task clearer to the children, but the teacher may not change the 
intent of the task . The teacher should alsc not give , any cues 
as to how the task is to be done . This is a test (although we 
never say this to the children) and should be treated as such. 

4. The teacher and aide (s) should carefully watch 'the children 
at work and become sensitive to when they need the instructions 
repeated once more. This is allowable, but again do not give the 
answer away. If questioned by a child the teacher or aide (s) 

should reply " .go ahead and do exactly as I have said. Do 

the very best you can.” 

5. As soon as a child appears to be finished 'the- teacher should 
check his work. If the child has been successful, the teacher is 

to write the ehildb name on his picture and send him to the aide to 
receive his candy. . 

6 . The question will arise as to what is the criterion 6f 
success, j-he child does not have to make. a perfect picture The 
teacher should be very flexible and lehient. That' is, if the task 



calls for the child to color a letter red., any red coloring in the 
appropriate letter is O.K. As long as no other object on the page 
ha s been colored , the child does not have to fill in all the letter . 
Similarly, if the child is asked to color a word, any coloring; of 
just that word,’ indicating!; that the child knows what you asked, is 
a success. Don’t be too strict. We want the children to succeed. 

7. The teacher is to check each child’s picture separately 
and send those children succeeding to the aide one at a time. 

8. The aide will collect the successful childrens’ pictures 
as the children are sent to her. The aide' will then give each of' 

a « , . • 

these children a bag of candy and emphasize that they are getting 
the candy because they have done such a good job. These children 
then go to another aide (it may be the same aide if only one (1) 
is available) who will inform each child of the choice of free play 
activities and allow the child to make his selection. 

9. The teacher during this time is explaining to the remaining 

0 * * * 

children- what should have been done with picture number three (3) 
This is the time to teach . The teacher then collects all the 
incorrectly marked picture, number 3’s and gives out picture number 

two (2), saying ” You almost got that picture right, but now 

lets try this one.” 

10. The teacher then reads the instructions for picture 
number 2, and goes through the same procedure as described above for 
picture number 3, 

11. The aides perform the same activities as described for 
picture number 3 (i.e., point #8) 

12. The teacher will explain to. the remaining children what 
• • • •'•■».. . * • . ' ■’ 
should have been done with picture number 2 and wili-then go' to 



picture number one (1) in the same fashion as presented above. 

13. The aides act as described above in picture number 3. 

14. At this time all the children should have been successful 
with at least one of the three pictures and received his candy and 
play activity, thus it is at this time that the aide or the teacher 
-indicates in the reinforcement record book the activity each child 
has selected. Next, the teacher or aide will obtain from the pictures 
collected the number of the picture each child was successful with 
and record this in the book. The pictures should then be placed 

in each child’s individual box, to be taken home later (the picture 
not the box) . Also recorded in the book at this time is the 
accumulated number of successful -lessons each child has completed 
up to this date. 

15. The teacher should look over the previous day’s book and 

determine which children' have received three (3) successes. So that 
upon their fourth (4th) success they will be given their choice of 
a toy . Thus, after every four successes the teacher is to begin 
recording from one (1) again. . * 

16. The teacher must select ahead of time the four (4) toys 
which will serve as prizes for a two-week period. These choices 
must be written on each record sheet for each lesson. Whenever 

a child has successfully completed 4 lessons the teacher or aide 
will allow him to chose his prize immediately after receiving his 
candy and prior to going to his play activity. The teacher or aide 
will record his choice in the record book. It may be clarified here, 
that the reason for keeping the same toys as choices for two weeks 

is to give the children a chance to pick another object which they 
saw and wanted the time before. 
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17. In the event that some children do not get any of the 
pictures correct, these children are not to receive any candy and arc 
to be told what activity they are to play at . Do not give these' 
children a choice. It must be emphasized here that tact must be 
used and the teacher must explain to these children why they have not 
-received the candy and choice of activity and stress the point that 
tomorrow they will get another cha.nce, 

18. This procedure of explaining why the children are being 

reinforced should be continued until the children understand how 
the system works . About two week s time should be sufficient for 
this understanding but if it is necessary to explain for each lesson 
by all means do so. • 

19. Anywhere within the above procedure the teacher may suggest 
to the children that they may color the rest of the pictures at home 
and can put them together to form a coloring book. Also, the 
teacher should mention that the candy may be eaten immediately or 
saved for a later time. 

20. The teacher is required to . change the choice of play 
activities once a week. This will provide additional variety for 
the children. 

V.* Concluding Remarks 

The experiment you are about to enter into is both significant 
and exciting. You will be contributing to the ever increasing body 
of knowledge concerned with how children learn. It is sincerely 
believed that the materials and training with which you have been 
provided in addition to your own ability and sensitivity will have 
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a significant effect upon the lives of your students. We wish you 
best of luck and would welcome any comments and/or opinions you may 
have pertaining to the implimentation of the reinforcement program. 




Barry J. Reinstein 
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Suggestions’ for .Classroom Arrant 



Specific areas vjithin room 

1. block corner 

2. doll corner 

3 . rug or circle (open place for story tell 

4. science corner 

5. painting area 



woodworking area 
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Suggested Tree Play Activities 
Incloor Equipment .(Manipulative Materials) 



1. feit board and felt pieces 

2 . pegboards 

3. puzzles 

4. blocks (units building kindergarten blocks ) 

5. wooden telephone 

6. ABC blocks 

7. beads 

8. small plastic bricks 

9. hammer - nail set 

10. parquetry set (a patterned wood inlay) 

11. wooden shoe 
12... dressing frames 

13. postal box 

14. counting frame. 

15. stacking discs . 1 

16. barrels (graduated size) 

17. clock 

18. knock-out bench 

19. pounding bench 

20. graduated cylinders 

21. dominoes 

22. divided puzzles 

23. sequence boards 

24. spinner board games 

25. lotto boards 

26. Lincoln logs 

27. block puzzles 

28. housekeeping dolls and dress-ups 

29. block (large, hollow) 

30. transportation toys 

•31. clay, play dough., paint, crayons 

32. water play 

33. paper maiche 

34. sand table 

35. paper cutting with scissors. 



36. 


books 


_ 






37. 


story telling 


or reading - 


listening time 


38. 


record player 












Outdoor 


Equipment 




1. 


jungle gym 




8. 


wheelbarrows 


2. 


sand box 


. 


9. 


small cars and trucks 


3. 


rubber saddle 


swings 


io. • 


doll carriages 


4. 


slide 


•— 


11. 


rocking boat 


5. 


tricycles of varying sizes 


12. 


balls 


6. 


peclal cars 




13. 


see- saw 


7 fc 


wagons 




14. 


balance board 
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Sug g ested Rewards 
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3 . watches 

2. boats, trucks, cars 

3. coloring books 

4. crayons 

• *' 5. clay 

6. animal erasers 

7 . ballons 

8 . marbles 1 

9. small plastic airplanes 

10 . baby dolls 

11. little girl bracelet and necklace 

12. parachutes 

13. binoculars 

14. western sets 

15. banjos 

16. skipping ropes 

17. monkeys on sticks 

18. kiddies stationery 

19. brave chief headdress 
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APPENDIX C 



National Evaluation Instruments 

I. Pre and Post Tests: 

1. Stanford-Dinet, long form 

2. Birch Response Style 

3. Inventory of factors affecting the Stanford-Binet 

4. Gumpgookies 

5. WPPSI - Animal House 

6. Revised Pre-School Inventory 

7. Family Interview 

8. Play Situation-Picture Board Sociometric Technique 

II. "Middle Measure" Tests: 

1. Classroom Observation of Substantive Curricular Input (OSCI) 

2. Post Observation Teacher Rating Scales (POT) 

III. Additional Child and Program Variable Instruments: 



1 . 


Characteristics of Teaching 


Staff 


2. 


Child Master Data Form 




3. 


Staff Member Information 




4. 


Class Registers 




5. 


Quality Control Standards 




6. 


Parent Consent Forms 




7. 


Class Facilities and Resources Inventory 


8. 


Class Parent Participation 1 


Record 


9. 


Center Parent Participation 


Record 


10. 


Medical /Dental Information 




11. 


Parent Inventory 









HEAD START INTERVENTION CHECK LIST 

APPENDIX D 

Region Treatment 

Center Book ■ Lesson 

Teacher . . Time: from to 

Monitor • __ Number of Children 

Date Number of Aides 



1. Adheres to 30-minutes specified time period per Buchanan 
lesson. (If less than 25 minutes or more than 35 
minutes, write amount of time in NO column.) 

2. - Adheres to specification of one Buchanan lesson per day. 

3. Adheres to correct sequencing of Buchanan lesson 
followed by supplement. 

4. Adheres to specified 20-minutes break following Buchanan 
lesson. (If less than 15 minutes or more than 25 
minutes, write time in NO column.) 

5. Adheres to allotted 30-minute time period for supplement- 
lesson. (If less than 25 minutes or more than 35 minutes 
write time in NO column.) 

6. Teaching was begun within 10 minutes of schedules time. 

If not , include explanation given by teacher for 
deviation from scheduled time in the COMMENTS section. 

7. Indicate which objectives of Buchanan lesson were 
covered by teacher and which were omitted. (Write in 
number of page corresponding to relevant objective.) 



COVERED 



OMITTED 



YF.S 


NO 



























8 . 



If applicable, indicate which activities of Swanson Supple- 
ment were covered by teacher. Which were omitted? If not 
applicable, check box to right. 

COVERED 



NOT 

APPLICABLE 




OMITTED 



COMMENTS ON PREPARATION AND ADMINISTRATION: 



9. 



Ill- AD START INTEITOjTiPN CHECK LIST 



Page 2 



10 . 

11 . 

12: 

13. 



14. 



15. 

16. 

17. 

18. 

19. 

20 . 



Region 



REINFORCEMENT 



Center __ Book Lesson 

Teacher _ Time * 



Monitor 



Number of Children 



Date 



Number of Aides 



Teacher follows correct sequencing of Buchanan lesson,, 
supplement, and reinforcement. 

Children are presented with test pictures immediately 
following completion of supplement lesson. 

Test pictures are handled appropriately, i.e., if 
child fails on first picture, next easiest picture 
is given, etc. 

Children are reinforced with M &-M*s following 
successful performance on a test picture. 

Teacher gives each child selection of activities 
immediately after, success with pictures. 

Time (15 to 20 minutes) is provided for child to 
engage in chosen activity following success on a 
test picture. 

Distribution of rewards takes place away from 
teaching area. 

Each student has his own place to keep and 
accumulate toys, etc. 

Reinforcement records are up to date. 

Reinforcement records are kept accurately. 



YES 


NO 












• 































Use the .space below to make any comments you feel are 
necessary and are not covered by the above items. 





APPENDIX E 



Committee on Educational Research 
University of South Carolina 



Head Start Evaluation and Research Center 
Examiner Evaluation Form 



Examiner’s Name Examiner’s No. 



Test Observed Observer 



Date Time spent observing (minutes) 

Directions: The observer is to place a letter rating (A, B, C, D) in the appropriate 

blank, designating in his judgment the examiner’s competence during the period of 
observation. Under "Comments," specific references should be made to relevant 
behavior, positive or negative, as appropriate. 

Rating Key: 

A - highly professional competence; near optimal performance for conditions; data 
valid. 

B - good overall competence, but with specific minor areas needing improvement; sati 
factory performance; data valid.' 

£ - competence only fair; inefficient procedures which jeopardize the validity of th 
data; this area requires special attention. 

I) - insufficient competence; inappropriate procedures which invalidate the data; un- 
acceptable performance. 

1. RAPPORT: Is the relationship established by the examiner conducive to valid re- 
sponses from the child? 

Comments 



2. MECHANICS: Is the examiner’s knowledge of and skill with materials and proce- 

dures sufficiently expert? 

Comments 



3. PROBING: Has the child’s capacity to respond correctly been maximally probed 

within the context of the item? 

Comments 



4. REINFORCEMENT: Are the examiner’s reactions to the child’s responses appropriate 

to the situation? 

Comments 



5. TEST-ORIENTED NEEDS: Does the examiner demonstrate a sensitivity to the charac- 

teristics of the child relevant to the test situation and adapt the administratic 
accordingly? 

Comments 



6. BIOLOGICAL NEEDS: Does the examiner demonstrate a sensitivity to fatigue, bore- 

dom, biological needs, etc., and take proper steps to alleviate them quickly? 
Comments 



7. In your judgment are the data obtained by this examiner acceptable? 
(A - Yes, B - Questionable, C - No) 

L£r|c 58 



APPENDIX F 

UNIVERSITY OF SOUTH CAROLINA 
COMMITTEE ON EDUCATIONAL RESEARCH 

REPORT ON TESTING CONDITIONS 

Date 

Observer 

Test 

Tester 

CENTER 

Name of Center City State 

TESTING SITE 

Type of Facility (room, porch, etc.) 

Location (center bldg., adjoining bldg., etc.) 



CONDITIONS 


Optimal 
or Good 


Fair but 
Acceptable 


Unacceptable 


Accessibility to 
Classroom 








Freedom from noise 








Privacy 








Working space 








Working surface, 
chairs, etc. 








Lighting 








Ventilation 








Temperature 








Cleanliness 


• 


1 


l 



COMMENTS 



OVERALL RATING OF CONDITIONS 
A - Optimal 

B - Good ' 

C - Fair but adequate (insignificant deleterious effects on data) . 
D - Conditions so poor that data are significantly impaired. 
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APPENDIX G 



1968-1969 Evaluation 
Sample Identification Questionnaire 




School of Education 



UNIVERSITY OF SOUTH CAROLINA 
COLUMBIA. S. C. 29208 



■ / 



July 30, 1968 



The University of South Carolina Evaluation and Research 
Center for Project Head Start is one of thirteen centers 
throughout the United States charged with the responsibility 
for performing research and evaluation functions designed to 
enhance the effectiveness of Head Start Programs. The South 
Carolina Center serves the states of Virginia, North Carolina, 
South Carolina, Georgia, and Florida. 

• During the 1968-69 year, the research emphasis in this 
region will be in the' area of language development. Thirteen 
Head Start Programs will be selected in the region to 
participate in the research. It has been suggested to us 
that your program might meet the criteria for inclusion in 
this sample and that you may be interested in participating. 

Essentially, the research will consist of implementing 
several approaches to language development in various locations 
and comparing the effectiveness of the programs. Children will 
be tested on several variables early in their Head Start 
experience and again near the end of their first year. 

Two teachers from each participating program (it "is 
recognized that one Head Start Center may have several programs 
or sub-units) will be involved in the investigation and some 
of these groups of two will receive training in the use of a 
given language program from personnel of the Evaluation and 
Research Center. In the case of one pair of teachers, the 
training will be approximately three weeks in duration. The 
teachers from the other twelve programs will receive consider- 
ably less training and in some cases none at all. 
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It is obvious that any time an ion-going educational program 
is involved in a research project, there is some inconvenience 
to those responsible for operating the program. Naturally if 
your program is included in 'the investigation, every effort will 
be made to make this inconvenience minimal. It does seem 
appropriate, however, to list as concisely as possible those 
items and activities which are required or will affect each 
participating program. These are' listed below: 

i‘ l.y Each child in one class of each participating 
1 program will be tested for approximately six 

hours in the fall and again for approximately 
• • six hours in the spring. No child will be 

tested for more than ninety minutes in one day. 

2. Two teachers in each participating program will 
. conduct a language development program in 

‘•accordance with instructions from the Evaluation 
and Research Center. In some cases this will 
. involve special training for the teacher which 
will be conducted by University of South 
Carolina personnel. ^ 

3. Because of the extensive individual testing 
noted above, it is necessary that one and 
perferably more rooms be available for testing 
purposes. It is necessary that the room(s) be 
equipped with a table and two chairs . 

4. In order to avoid delays and conserve resources, 
children must be available when needed for 
testing during the school day. 

5. In addition to tests administered to each child, 
additional information related to the classroom 
and to teaching personnel will be collected. 

6. An observer will visit one of the two participating 
classes in each program on an average of twice per 

" week. 

7. Less frequently, other observers will visit the 
Center for supplementary information. 



i 
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In order to determine if programs conducted by you?' Center 
meet certain other criteria for inclusion in the research 
project, a questionnaire is enclosed. The questions refer 
to the Head Start classes which will begin in late summer or 
early fall of 1968. As some of the information requested 
cannot be known at this time, it will be necessary for you 
to submit estimates which are as accurate as you can make 
them now. We will sincerely appreciate' your completing the 
questionnaire and returning it to this office by August 5, 

1968. \ • ' 



We are most interested in having your program participate 
in the investigation and we believe that our combined efforts 
may lead to improved programs of language development in pre- 
school educational programs. 

Thank you for your consideration. 




Director of Research Operations 
Committee on Educational Research 



Enclosure 



* 
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UNIVERSITY OF SOUTH CAROLINA 
REGIONAL EVALUATION AND RESEARCH CENTER 
' FOR PROJECT HEAD START 

1968-1969 EVALUATION ■ 

SAMPLE IDENTIFICATION QUESTIONNAIRE 

NAME OF CENTER: ; ' ' Office Tel. No. 

ADDRESS: •' __ _ 

v . ^ ' 2 •• 

CENTER DIRECTOR: • • Home Tel. No. _ 

-ADDRESS: • ■ ’ ' 



Please -place an "X 11 in the appropriate box for each question. If the 
question calls for a numerical quantity, place the appropriate number 
in the box. 



If any of the iterps need clarification in order for you to answer them, 
please feel free to call Mr. Boyd LeGrand or Mr. Robert Branham collect: 
803/777-8108, Columbia, South Carolina. 



Governing Agency: 



O Community Action Program 



□ Board of Education 



I. LANGUAGE DEVELOPMENT PROGRAM 



* Y es No Number 

□ a 

□ tn 

□ 



1. Is your Head Start Center interested in 
adopting a new language development program? 

2. Are there at least two teachers in your Head 
Start Center who would be interested in 
learning a new language development program? 

3. If the answer to #2 is "yes,” how many 
teachers would be interested in learning 
a new language development program? 
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